This invention relates to computer programs. In particular, this invention relates to a method of developing and testing computer programs, and computer software for implementing the method.
The use of computers for home, commercial, industrial and academic applications has expanded exponentially in the past two decades, to the point where computers are now amongst the most versatile tools known to mankind. In large part this is due to software development, which can be credited for the versatility and complexity of today""s myriad computer functions.
The software development life cycle (SDLC) has conventionally consisted of three phases:
1. Determining system requirements through information from the intended user, which typically involves a detailed interrogation of the user to determine the user""s objectives, and extrapolation from the developer""s understanding of these objectives to determine the desired system behavior under the circumstances contemplated by the end user;
2. Constructing design parameters which most efficiently achieve the system requirements in the context of data entry, data processing and user interface; and
3. Implementing the design requirements by developing program code intended to meet the design parameters. This phase typically concludes with extensive testing of the program code designed to uncover errors introduced during the implementation phase.
This software development methodology was adequate when software engineering was in its infancy, because of the relative simplicity of the software and hardware components of most computer systems. However, it has led to a xe2x80x98create, develop and testxe2x80x99 software development approach, which is unsuitable and inefficient in the context of today""s complex computer systems and networks.
This approach to software development presents problems at all three levels.
First, errors are introduced at each phase. Because there is no universally accepted formal language available to describe the system requirements, the system developer""s understanding of the user""s requirements is subject to the vagaries and ambiguities of the informal language (for example, English) in which the requirements are conveyed. There is no common frame of reference from which it can be determined whether the system requirements are consistent with the user""s intent, and invariably due to the diversity of human experience and understanding, in many respects developer""s understanding will not be consistent with the user""s intent. Thus, errors are introduced into the first phase of software development by inaccurate translation of the user""s requirements into system requirements, and these errors are perpetuated throughout the remainder of the SDLC.
Errors are similarly introduced into the design phase, based on the extrapolation of the system requirements into design requirements, again due to the informality of the language used to represent these requirements at both levels. These errors are cumulative with the errors introduced in the system requirements phase, and all of these errors become latent limitations of the system design phase. At this stage none of these errors are manifest, because the end user typically does not participate in the SDLC after the first phase, and the end user has thus not received any feedback which would indicate that the user""s requirements have been misunderstood and/or improperly translated into the design level.
Errors are introduced into the implementation phase in the same fashion, so that by the time the creation of the actual program code commences the initial system requirements as contemplated by the user are contaminated by three levels of errors. In a complex computer system, immense time and effort is required to expose and correct these errors, and there is the possibility (in fact, a high probability) that errors will be overlooked and the software will be released for use with errors still in it. This is at best an annoyance to the end user, and at worst can be catastrophic; for example in the case of computer systems for diagnostic or therapeutic use in hospitals, systems controlling functions in nuclear facilities, air traffic control systems etc.
Moreover, computer programs are incapable of resolving inherent inconsistency, incompleteness, incompatibility and ambiguity. These deficiencies can arise at each stage of the SDLC, but the system requirements stage is acutely susceptible to the injection of such errors. The end user may have a detailed understanding of their own requirements, but can rarely anticipate all possible results of any particular data entry or processing scenario, which leads to incompleteness and ambiguity errors; in the design requirements phase it is usually impossible to appreciate all possible ways and environments in which the system will be used, which leads to incompleteness and incompatibility errors; and in complex systems system behavior may be reliant upon thousands or even millions of processing steps, with many possible results, which can lead to ambiguity and inconsistency errors. Complex computer programs are chaotic systems, in which a small error occurring early in a computational process can result in unpredictable and unproductive, or even counterproductive, system behavior.
Rectification of these types of errors often involves creating rectifying code which is designed solely to xe2x80x9cundoxe2x80x9d the errors, and which has no other legitimate purpose. This serves only to increase the complexity of the system, and is almost entirely a consequence of the xe2x80x98create, develop and testxe2x80x99 approach to software development. The cost and difficulty of rectifying an error in the implementation phase of the SDLC is orders of magnitude greater than the cost and difficulty of rectifying an error in the requirements phase or the design phase of the SDLC. Moreover, identification and rectification of errors in the pre-implementation phases considerably reduces the need for rectifying code in the implementation stage.
The testing of such systems is also problematic. According to conventional software testing methods, an analysis is conducted on the completed computer program and one or more test programs are derived from the original program. Being derivatives of the original program, however, the test programs will necessarily contain the same errors as the original program. Thus, the results of further analysis using these derivative test programs may be satisfactory, or even excellent, but only because the test programs themselves are flawed in the same manner as the original program and therefore latent errors injected into the various stages of the SDLC cannot be detected by the tests. There is no effective way to test the tests, because there is no standard methodology used to create the tests and therefore no way of validating the usefulness of the results.
Moreover, these tests are created on an ad hoc basis by each software developer in the context of the particular computer program under consideration, without regard for the compatibility of the original computer program with software and systems created by other developers. There has heretofore been no widely used verification or validation model which can be applied universally to software created by different developers. These problems are exacerbated by the explosive popularity of the Internet, which endeavors to accommodate software of many different types and platforms. Latent incompatibility problems have become so pervasive as to be commonplace, and are actually expected by most Internet users.
The present invention overcomes these disadvantages by providing a method of modeling a computer program at each stage of the SDLC, and a computer-aided software testing and quality assurance method using a universal verification and validation model which can be applied to any type of software at any stage during the SDLC. The invention also provides software for implementing the verification and validation functions of the invention. The method of the invention provides a means for ensuring and verifying compatibility and fidelity between the original system requirements and the final software product, and for deriving tests which provide an accurate analysis of the system that takes into account the original system requirements. Because the method of the invention can be applied universally to any hardware/software system, its widespread use would also eliminate incompatibility between different software products, platforms and operating systems.
The invention accomplishes this by implementing a validation and verification process in each phase of the SDLC which represents the system requirements as a series of positions and transitions associated with natural language descriptions of the system requirements. In the preferred embodiment this involves creating a formal model of each phase of the SDLC to eliminate intra-phase inconsistencies and ambiguities and ensure completeness and compatibility, before proceeding to the next phase of the SDLC. These models are preferably based on classical Petri nets and variations or extensions thereof. In the preferred embodiment the models created in each phase are cross-reconciled, to eliminate inter-phase degradation by errors which may have been injected in the transition between SDLC phases.
Testing programs for the finished software product are derived both from the parameters at each phase of SDLC model, and from the verification and validation models created at each stage of the SDLC according to the invention, using Petri net analysis techniques. This ensures that analyses of the software product are founded not only in the programmer""s conception of the system at the implementation stage, but also on the end user""s intended system requirements and the design parameters derived therefrom. The invention accordingly provides a test methodology which incorporates factors extraneous to the finished software product, to generate analyses which bear on the system requirements and the design parameters, and not just intrinsic features of the finished software product.
The invention thus provides methods for automatically generating models representing a computer program and for automatically generating tests of the program.
According to the invention a method of developing a model of a computer program on a system level can be implemented in different ways, depending upon the original representation of the system requirements.
A manual embodiment of the method of model development represents the system requirements, provided in a natural language (such as English), as a set of positions, a set of transitions and a marking vector of an extended Petri net model; maps the system requirements in corresponding tables of positions and transitions with selected transition and position designations; and tags one or more significant positions (or transitions).
In an automatic embodiment of the method of system requirements model development, the system requirements are represented as a table of causes and effects. The system requirements are automatically transferred from this table to an extended Petri net model; mapped from the table of causes and effects to corresponding tables of positions and transitions; and one or more designated significant positions or transitions are tagged.
In an automatic embodiment of the method of model development based on a xe2x80x9cUse Casexe2x80x9d model, the Use Case is translated to an extended Petri net model as a basic flow of positions and transitions, and alternative flows of positions and transitions connected to positions or transitions in the basic flow. The method maps the system requirements of the basic flow in a table of positions with position designations; tags one or more significant positions in the basic flow including any conditional positions; maps the system requirements of the alternative flows in a table of positions with position designations; and creates a reference designation associating the alternative flows with their respective conditional events in the basic flow.
The modeling process is independent of the method of model development. A marking vector defines the initial conditions of the system, and the modeling process is implemented according to the rules of the selected Petri net variation. The tags provide for external control of the modeling process, allowing for artificial activation of corresponding redundant positions, depending on the portions of the program being modeled, and determine the most critical states for the testing process. The modeling results can then be provided in terms of requirements, which may be expressed in a natural language.
This same modeling method is also used at the design requirements level, for example using algorithms and mathematical expressions, and at the implementation level, for example using algorithms and program code.
The universal automated software testing method of the invention represents the system requirements, design requirements or programs as an extended Petri net model, isolates significant positions by reverse analysis, creates a prototype table of the sequence of states covering the part of the model under consideration, and from the prototype table and table of positions generates tests of a required level expressed in terms of natural language, mathematical expressions or data. These tests can be used for verification and validation, either directly or with some additional information.
For verification of the current level requirements the model representation is tested using tests developed from the previous level. For example, design level requirements are verified using tests generated at the system level. Validation of the software product is effected by conventional testing processes using higher level tests.
In the preferred embodiment the invention also provides tools, comprising universal testing software, for implementing the testing method of the invention.
The present invention thus provides a method of developing a model of a computer program having program code for implementing specified requirements, the method comprising the steps of a. applying position designations to conditions in the requirements, b. mapping the positions in a table of positions relating the position designations to the corresponding conditions; c. identifying one or more significant positions which are positions defining specific points for test development, d. adding as an input to each transition immediately following a significant position a redundant position having no input transition, the state of which controls activation of its output transition, and e. mapping the redundant positions in the table of positions.
The invention further provides a computer program product for use with a computer, the computer program product comprising a computer usable medium having computer readable program code means embodied in said medium for generating a model of a computer program having program code designed to implement specified requirements comprising a series of conditions, each condition having a position designation, and one or more significant positions which are positions defining a segments of the model which can be analyzed, and having added as an input to each transition immediately following a significant position a redundant position having no input transition, the state of which controls activation of its output transition, said computer program product having a. computer readable program code means for mapping the positions in a table of positions relating the position designations to the corresponding conditions, b. computer readable program code means identifying one or more significant positions which are positions defining specific points for test development, and c. computer readable program code means for mapping the redundant positions in the table of positions.
The present invention further provides a method of test development for a computer program having program code and designed to implement requirements comprising conditions and events, to verify concordance between the computer program and the requirements, the method comprising the steps of a. mapping the conditions to a set of positions comprising at least one starting position which has no input, at least one terminal position which has no output, and intermediate positions and transitions connected between the starting position and the terminal position, and b. conducting a reverse analysis of a portion of the requirements from a terminal position to generate from the set of positions a table of positions comprising, in sequence, the terminal position, positions between the terminal position and the starting position, and the starting position, the table comprising an expression of positions representing the corresponding requirements.
A further aspect of the invention includes the method steps of applying transition designations to events in the requirements and mapping the transitions in a table of transitions relating the transition designations to the corresponding events, and computer readable program code means for same.
The invention accordingly provides a unified method for analyzing complex software systems at different levels of system representation, to resolve many quality assurance and testing problems inherent in conventional methodologies.